Utilizing augmented reality to virtually trace cables

ABSTRACT

Systems and methods for utilizing Augmented Reality (AR) processes to track cables among a tangled bundle of cables are provided. An AR method, according to one implementation, includes a step of obtaining an initial captured image showing a bundle of cables. The AR method also includes the step of processing the initial captured image to distinguish a selected cable from other cables of the bundle of cables. Also, the AR method includes displaying the initial captured image on a display screen while visually augmenting an image of the selected cable to highlight the selected cable with respect to the other cables.

TECHNICAL FIELD

The present disclosure generally relates to electrical and opticalcables in a network. More particularly, the present disclosure relatesto utilizing augmented reality to superpose visual features on an imageof a selected cable to distinguish the selected cable from a bundle ofcables.

BACKGROUND

In many telecommunications environments, it can be difficult to manage alarge number of cables within a limited space. Since a typicaltelecommunication service provider may be capable of handle hundreds orthousands of customers with various configurations, offering variousservices to the customers may require thousands of cables among thenetwork equipment.

As a result of connecting multiple cables to a confined rack or chassisof a data center, server room, or other network system location, someissues may arise. First, due to poor equipment installation ormaintenance, the cables may be arranged in a disorderly, unorganizedmanner. This can create a tangled mess, which may be referred tocolloquially as a “rat nest” or a “spaghetti” configuration. Over time,this tangled bundle of cables may cause costly maintenance issues.

Another issue with this disorganized cabling arrangement is that it maybe difficult for network technicians to follow the path of a certaincable. For instance, if a technician wishes to observe the endconnections of a cable or to replace a faulty cable, it can be difficultto follow the particular cable from one end to the other. Even with aclean cable organization, it can be difficult to confirm where a cablestarts or ends only by looking at it. Correcting the above issues inthese conventional systems requires maintenance downtime involvingmanual manipulation and specialized tools. The longer it takes for thetechnician to follow cables during this downtime, the greater thenegative impact on customers.

Therefore, there is a need in the field of cable management systems toenable a technician to track cables more easily when a group of cablesare bundled or tangled together in a limited space.

BRIEF SUMMARY

The present disclosure is directed to Augmented Reality (AR) systems, ARmethods, portable devices with AR capabilities, smart phoneapplications, and non-transitory computer-readable media for performingAR processes to assist a user with a task of following or tracking thepath of a selected cable arranged with a plurality of other cables. Froman image of a bundle of cables displayed on a display screen of a userdevice, a user can select one of the cables that he or she wishes totrack. The AR application is configured to use image processingfunctionality to determine which cable sections belong to the selectedcable and which cable sections belong to other cables. Then, the ARapplication can add virtual image features to the displayed image tohighlight or emphasize the selected cable so that the user does not losefocus on the selected cable. For example, even if the user blinks andloses track of the selected cable, the AR application is able to remainfocused on the selected cable and continuously highlight the selectedcable as the user moves the portable image-capture device along thelength of the cables.

According to one implementation, an AR method may include a step ofobtaining an initial captured image showing a bundle of cables. The ARmethod also includes processing the initial captured image todistinguish a selected cable from other cables of the bundle of cables.Also, the AR method includes the step of displaying the initial capturedimage on a display screen while visually augmenting an image of theselected cable to highlight the selected cable with respect to the othercables.

Additional embodiments of the AR systems and methods may be provided.For example, the AR method may further includes the steps of: a)receiving a next captured image, b) comparing the next captured imagewith a previous captured image, c) using a tracking technique toidentify at least a section of the selected cable in the next capturedimage, d) distinguishing the selected cable shown in the next capturedimage from the other cables, e) updating the display screen to displaythe next captured image while visually augmenting a next image of theselected cable to highlight the selected cable with respect to the othercables, and f) repeating the receiving, comparing, using,distinguishing, and updating steps one or more times in a continuousmanner until the user decides to stop the application. Also, the step ofcomparing the next captured image with the previous captured image mayinclude determining differences between the next captured image theprevious captured image when an image capture device captures the nextcaptured image and previous captured image from different viewpoints orwhen the at least one cable of the bundle of cables is moved.

Furthermore, the step of processing the initial captured image mayinclude one or more procedures selected from the group of proceduresconsisting of an object recognition procedure, a line-crossing analysis,a line representation analysis, a point detection procedure, a colorfiltering procedure, a Canny-edge detection procedure, a morphologicalprocedure, a dilation operation, an erosion operation, a skeletonizationprocedure, and a corner deletion procedure. Processing the initialcaptured image may also include tracking paths of each cable of thebundle of cables in one or two directions. Processing the initialcaptured image may include mapping each cable of the bundle of cables ina three-dimensional map.

The step of obtaining the initial captured image may include receivingthe initial captured image from a camera of a portable user device,whereby the display screen may be associated with the portable userdevice. The portable user device, for example, may be one of a smartphone, a tablet, eyewear, and a head-up display. The AR method caninclude the step of receiving a user input for identifying the selectedcable of the bundle of cables.

The step of visually augmenting the image of the selected cable mayinclude the step of superposing virtual image features on the image ofthe selected cable. Superposing virtual image features may include thestep of adding color or animation features to the image of the selectedcable. The step of visually augmenting the image of the selected cablemay further include displaying one or more visual features for showingportions of the selected cable that are hidden from view. The AR methodmay further include the step of using machine learning to distinguishthe selected cable from the other cables.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is illustrated and described herein withreference to the various drawings. Like reference numbers are used todenote like components/steps, as appropriate. Unless otherwise noted,components depicted in the drawings are not necessarily drawn to scale.

FIG. 1 is a block diagram illustrating a portable device for visuallytracing a selected cable from a bundle of cables using augmentedreality, according to various embodiments of the present disclosure.

FIG. 2 is a flow diagram illustrating a first process for visuallytracing a selected cable using augmented reality, according to variousembodiments of the present disclosure.

FIGS. 3A and 3B are images of cables and trace selection points relatedto the image capture and cable selection steps shown in FIG. 2,according to various embodiments.

FIGS. 4A-4E are images of cables and visual highlighting featuresrelated to the cable visualization step shown in FIG. 2, according tovarious embodiments.

FIG. 5 is a flow diagram illustrating a cable identifying sub-routine ofone of the steps of the process of FIG. 2, according to variousembodiments.

FIG. 6 is an image with respect to a color filtering step of thesub-routine of FIG. 5, according to various embodiments.

FIG. 7 is an image with respect to Canny-edge detection, dilation, anderosion steps of the sub-routine of FIG. 5, according to variousembodiments.

FIG. 8 is an image with respect to a skeletonization step of thesub-routine of FIG. 5, according to various embodiments.

FIG. 9 is an image with respect to a corner detection step of thesub-routine of FIG. 5, according to various embodiments.

FIG. 10 is an image with respect to line and point detection steps ofthe sub-routine of FIG. 5, according to various embodiments.

FIG. 11 is an image with respect to a cable section tracking step of thesub-routine of FIG. 5, according to various embodiments.

FIG. 12 is an image with respect to a cable section joining step of thesub-routine of FIG. 5, according to various embodiments.

FIG. 13 is a flow diagram illustrating a second process for visuallytracing a selected cable using augmented reality, according to variousembodiments of the present disclosure.

FIG. 14 is a flow diagram illustrating another process for visuallytracing a selected cable using augmented reality, according to variousembodiments of the present disclosure.

DETAILED DESCRIPTION

The present disclosure relates to systems and methods for utilizingAugmented Reality (AR) to display real images while also providingadditional virtual image features. More specifically, the systems andmethods of the present disclosure are configured to allow a user toselect a specific cable from a bundle of cables arranged in a smallspace (e.g., terminated at a back panel of rack-mounted networkequipment deployed in a data center). Since cable bundles can oftenbecome a tangled mess, the present embodiments are configured to useimage processing techniques or algorithms to analyze an image of thebundled cables and distinguish the selected cable from the other cables,regardless of which cables are in the foreground, in the background,hidden behind some cables, etc. For example, the systems and methods areable to determine a somewhat straight path of each path, which may bebased on normal rigidity or bendability characteristics of cables. Oncethe image analysis processes of the present disclosure locate theselected cable and distinguish it from the other cables, the ARprocedures may include the process of adding virtual image features(e.g., highlighted traces) superposed over the image of the selectedcable. In this way, the user can more easily keep track of the selectedcable.

AR is currently used in many applications, such as in manufacturing,maintenance, repair, warehouse operation, aircraft manufacturinginvolving the connection of multiple electrical components, etc. ARinvolves an interactive experience of a real-world environment, where acamera view of real-world objects is enhanced by computer-generatedgraphics overlaid on the real-world images. Different vendors haveintroduced various AR development platforms, making it possible forusers of smart phones or tablets to access AR applications.

The present disclosure describes systems and methods to assist networktechnicians, cable installers, aircraft manufacturers, and other userswho work in an environment where it may be necessary to handle or trackmultiple electrical, optical, electro-optical cables arranged in a smallarea. The AR systems and the methods assist the user with wiringpredefined paths already provided by a formboard, and, in short, may beconfigured to graphically provide wiring assembly instructions.Presently, conventional AR systems do not allow a user to select a cablefrom a bunch of cables and use AR to highlight the selected cable on adisplay screen, as described in the present disclosure. Thus, thepresent disclosure provides AR systems and methods to help a worker todynamically track the path of a selected cable among a “rat nest” ofmultiple cables, by virtually highlighting, on the user's displayscreen, the path of the selected cable.

The AR systems and methods may be associated with suitable applicationsthat run on portable devices (e.g., smart phones, tablet computers,etc.). According to various embodiments of the present disclosure, theportable device may include an image capture device (e.g., camera, videocamera, etc.) for capture an image or a series of images that can beprocessed by AR functionality as describe in the present disclosure tofollow the path of the cables. Also, the portable device may include adisplay screen for showing the combination of real-world images capturedby the image capture device plus virtually added or augmented imagefeatures on or with the real image.

The application utilizes AR to help a worker dynamically track the pathof a selected cable among a bundle (also referred to as a “rat nest,”“spaghetti,” etc.) of multiple cables. The cable to be tracked isselected on the display screen by the worker and the AR processes areconfigured to graphically highlight this cable on the display screen.For example, the highlighting feature may include adding a color oranimations on the live (real) image rendered by the portable device. TheAR processes are also configured to dynamically update new images as theviewpoint of the user's portable device is moved. Therefore, the systemsand methods of the present disclosure are configured to keep track ofthe location of the selected cable even while the position of the camerachanges or even while the cable are moved. For example, in somesituations, it may be helpful for the user to move other cables out ofthe way if they block the view of the selected cable.

In some embodiments, a user can select equipment and then thecable-following application may be configured to retrieve informationabout cables that were previously identified. The application can theneither show identification information about the cables on the displayscreen, which may be overlaid on the live image rendered by the portabledevice. The application can also point in the direction that the cablesare directed on the live image. If the cable destination is not visiblein the rendered image, the application may point the user to thedestination where it is located using either an aerial view, see-throughimages, arrows, or other visual effects overlaid on the live imageindicating where to position the portable device and/or which directionto orient the portable device.

The AR processes can help a worker dynamically track the path of aselected cable among multiple cables by highlighting the path of theselected cable on the display screen. The AR processes can distinguishthe selected cable from among the other cables. For example, theseprocesses may include line-crossing analysis on two-dimensional (2D)images. Also, the AR processes can dynamically track cable when theviewpoint and/or orientation of the image capture device changes and/orwhen the cables are moved. Thus, these processes may be configured tokeep track of the selected cable path even if points along the cable arelost.

There has thus been outlined, rather broadly, the features of thepresent disclosure in order that the detailed description may be betterunderstood, and in order that the present contribution to the art may bebetter appreciated. There are additional features of the variousembodiments that will be described herein. It is to be understood thatthe present disclosure is not limited to the details of construction andto the arrangements of the components set forth in the followingdescription or illustrated in the drawings. Rather, the embodiments ofthe present disclosure may be capable of other implementations andconfigurations and may be practiced or carried out in various ways.Also, it is to be understood that the phraseology and terminologyemployed are for the purpose of description and should not be regardedas limiting.

As such, those skilled in the art will appreciate that the inventiveconception, upon which this disclosure is based, may readily be utilizedas a basis for the designing of other structures, methods, and systemsfor carrying out the several purposes described in the presentdisclosure. Those skilled in the art will understand that theembodiments may include various equivalent constructions insofar as theydo not depart from the spirit and scope of the present invention.Additional aspects and advantages of the present disclosure will beapparent from the following detailed description of exemplaryembodiments which are illustrated in the accompanying drawings.

FIG. 1 is a block diagram illustrating an embodiment of a portabledevice 10 for visually tracing a selected cable from a bundle of cablesusing augmented reality. In the illustrated embodiment, the portabledevice 10 may be a digital computing device that generally includes aprocessing device 12, a memory device 14, Input/Output (I/O) interfaces16, an external interface 18, and a database 20. It should beappreciated that FIG. 1 depicts the portable device 10 in a simplifiedmanner, where some embodiments may include additional components andsuitably configured processing logic to support known or conventionaloperating features. The components (i.e., 12, 14, 16, 18, 20) may becommunicatively coupled via a local interface 22. The local interface 22may include, for example, one or more buses or other wired or wirelessconnections. The local interface 22 may also include controllers,buffers, caches, drivers, repeaters, receivers, among other elements, toenable communication. Further, the local interface 22 may includeaddress, control, and/or data connections to enable appropriatecommunications among the components 12, 14, 16, 18, 20.

According to some embodiments, the I/O interfaces 16 may include one ormore image capture devices (e.g., cameras, video cameras, etc.) forcapturing images of a plurality of cables in a bunch or bundle. Also,the I/O interfaces 16 may include one or more display devices, displayscreens, or other suitable output devices for visually depicting thecaptured images. Also, in accordance with the AR processes described inthe present disclosure, the display devices or display screens of theportable device 10 may be configured to show additional visual featuresthat may be generated by the AR processes to help highlight the selectedcable with respect to the other (non-selected) cables in the real-worldview.

The portable device 10 may include an augmented-reality cable-trackingprogram 24, which may be configured to utilize AR to track a selectedcable along its path, even when the viewpoint of the captured imagechanges and the images are changed. The augmented-reality cable-trackingprogram 24 may be configured to automatically refresh the newly capturedimages on the display screen to provide substantially real-time feedbackto the user. The augmented-reality cable-tracking program 24 may beimplemented in software and/or firmware and stored in the memory device14, as depicted in FIG. 1. In other embodiments, the augmented-realitycable-tracking program 24 may be implemented entirely or partially inhardware in the processing device 12.

According to some embodiments, the external interface 18 and databased20 may be omitted from the portable device 10. For example, in theseembodiments, the portable device 10 may be used as a stand-alone devicefor providing AR cable-following functionality for the user. However, inother embodiments, the external interface may be configured tocommunicate with external devices (e.g., via cellular signals, Wi-Fi,Bluetooth, etc.). Communication with external devices may be providefunctionality to enable cable-following images to be provided to aremote device. For example, the portable device 10 may be used as a“gopher” device for capturing images, which can then be communicated toa remote device (e.g., a device associated with a network operator whomay then analyze the images, provide instruction about directing thegopher, etc.).

It should be appreciated that the processing device 12, according tosome embodiments, may include or utilize one or more generic orspecialized processors (e.g., microprocessors, CPUs, Digital SignalProcessors (DSPs), Network Processors (NPs), Network Processing Units(NPUs), Graphics Processing Units (GPUs), Field Programmable Gate Arrays(FPGAs), semiconductor-based devices, chips, and the like). Theprocessing device 12 may also include or utilize stored programinstructions (e.g., stored in hardware, software, and/or firmware) forcontrol of the portable device 10 by executing the program instructionsto implement some or all of the functions of the systems and methodsdescribed herein. Alternatively, some or all functions may beimplemented by a state machine that may not necessarily include storedprogram instructions, may be implemented in one or more ApplicationSpecific Integrated Circuits (ASICs), and/or may include functions thatcan be implemented as custom logic or circuitry. Of course, acombination of the aforementioned approaches may be used. For some ofthe embodiments described herein, a corresponding device in hardware(and optionally with software, firmware, and combinations thereof) canbe referred to as “circuitry” or “logic” that is “configured to” or“adapted to” perform a set of operations, steps, methods, processes,algorithms, functions, techniques, etc., on digital and/or analogsignals as described herein with respect to various embodiments.

The memory device 14 may include volatile memory elements (e.g., RandomAccess Memory (RAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM),Static RAM (SRAM), and the like), nonvolatile memory elements (e.g.,Read Only Memory (ROM), Programmable ROM (PROM), Erasable PROM (EPROM),Electrically-Erasable PROM (EEPROM), hard drive, tape, Compact Disc ROM(CD-ROM), and the like), or combinations thereof. Moreover, the memorydevice 14 may incorporate electronic, magnetic, optical, and/or othertypes of storage media. The memory device 14 may have a distributedarchitecture, where various components are situated remotely from oneanother, but can be accessed by the processing device 12.

The memory device 14 may include a data store, database (e.g., database20), or the like, for storing data. In one example, the data store maybe located internal to the portable device 10 and may include, forexample, an internal hard drive connected to the local interface 22 inthe portable device 10. Additionally, in another embodiment, the datastore may be located external to the portable device 10 and may include,for example, an external hard drive connected to the Input/Output (I/O)interfaces 16 (e.g., SCSI or USB connection). In a further embodiment,the data store may be connected to the portable device 10 through anetwork and may include, for example, a network attached file server.

Software stored in the memory device 14 may include one or moreprograms, each of which may include an ordered listing of executableinstructions for implementing logical functions. The software in thememory device 14 may also include a suitable Operating System (O/S) andone or more computer programs. The O/S essentially controls theexecution of other computer programs, and provides scheduling,input/output control, file and data management, memory management, andcommunication control and related services. The computer programs may beconfigured to implement the various processes, algorithms, methods,techniques, etc. described herein.

Moreover, some embodiments may include non-transitory computer-readablemedia having instructions stored thereon for programming or enabling acomputer, server, processor (e.g., processing device 12), circuit,appliance, device, etc. to perform functions as described herein.Examples of such non-transitory computer-readable medium may include ahard disk, an optical storage device, a magnetic storage device, a ROM,a PROM, an EPROM, an EEPROM, Flash memory, and the like. When stored inthe non-transitory computer-readable medium, software can includeinstructions executable (e.g., by the processing device 12 or othersuitable circuitry or logic). For example, when executed, theinstructions may cause or enable the processing device 12 to perform aset of operations, steps, methods, processes, algorithms, functions,techniques, etc. as described herein according to various embodiments.

The methods, sequences, steps, techniques, and/or algorithms describedin connection with the embodiments disclosed herein may be embodieddirectly in hardware, in software/firmware modules executed by aprocessor (e.g., the processing device 12), or any suitable combinationthereof. Software/firmware modules may reside in the memory device 14,memory controllers, Double Data Rate (DDR) memory, RAM, flash memory,ROM, PROM, EPROM, EEPROM, registers, hard disks, removable disks,CD-ROMs, or any other suitable storage medium.

Those skilled in the pertinent art will appreciate that variousembodiments may be described in terms of logical blocks, modules,circuits, algorithms, steps, and sequences of actions, which may beperformed or otherwise controlled with a general purpose processor, aDSP, an ASIC, an FPGA, programmable logic devices, discrete gates,transistor logic, discrete hardware components, elements associated witha computing device, controller, state machine, or any suitablecombination thereof designed to perform or otherwise control thefunctions described herein.

The I/O interfaces 16 may be used to receive user input from and/or forproviding system output to one or more devices or components. Forexample, user input may be received via one or more of a keyboard, akeypad, a touchpad, a mouse, and/or other input receiving devices.System outputs may be provided via a display device, monitor, UserInterface (UI), Graphical User Interface (GUI), a printer, and/or otheruser output devices. I/O interfaces 16 may include, for example, one ormore of a serial port, a parallel port, a Small Computer SystemInterface (SCSI), an Internet SCSI (iSCSI), an Advanced TechnologyAttachment (ATA), a Serial ATA (SATA), a fiber channel, InfiniBand, aPeripheral Component Interconnect (PCI), a PCI eXtended interface(PCI-X), a PCI Express interface (PCIe), an InfraRed (IR) interface, aRadio Frequency (RF) interface, and a Universal Serial Bus (USB)interface.

The external interface 18 may be used to enable the portable device 10to communicate over a network, the Internet, a Wide Area Network (WAN),a Local Area Network (LAN), and the like. The external interface 18 mayinclude, for example, an Ethernet card or adapter (e.g., 10BaseT, FastEthernet, Gigabit Ethernet, 10 GbE) or a Wireless LAN (WLAN) card oradapter (e.g., 802.11a/b/g/n/ac). The external interface 18 may includeaddress, control, and/or data connections to enable appropriatecommunications on a network.

FIG. 2 is a flow diagram illustrating a first embodiment of a process 30for visually tracing a selected cable using augmented reality. In thisembodiment, the process 30 include a first step of capturing a firstimage, as indicated in block 32. Capturing the image may includecapturing a single image, capturing multiple images over a short time(e.g., video), allowing a user to utilize an image capture device (e.g.,camera, video camera, etc.) to capture one or more images where one maybe chosen as a representation of what the user may be trying to do.

The process 30 further includes receiving a start point on the imagefrom a user, as indicated in block 34. From the image captured in block32, the start point is selected by the user. This start point maypreferably be a point clearly on one of the cables shown in the capturedimage and clearly visible in the frame of the image (i.e., not hiddenfrom view). The process 30 also includes identifying a selected cablewithin the image, as indicated in block 36. This step (i.e., block 36)may include one or more various image processing steps for identifyingthe selected cable. Also, the image processing of block 36 is furtherdescribed with respect to FIG. 5 below.

After processing the image to identify the selected cable (block 36),the process 30 includes the step of visualizing (i.e., displaying,depicting, representing, etc.) the selected cable on the image, asindicated in block 38. For example, this step may include addingaugmented visual cues or other highlighting or emphasis-type features todisplay the selected cable in a different way with respect to the othercables in view in the image.

In this embodiment, the process 30 further include a condition diamond40 for determining whether an indication has been received from the userindicating that the user wishes to stop. If the user wishes to stop, theprocess 30 comes to an end. Otherwise, if no “stop” indication has beenreceived, the process 30 proceeds to block 42. As indicated in block 42,the process 30 includes capturing a new image at a subsequent intervalof time. It may be desirable to be able to handle multiple images persecond to allow a continuous transition from one image to the next. Ifthe user moves the portable device (e.g., portable device 10) tooquickly, for example, a discontinuity in sequential images may result ina loss of the virtual identification of the selected cable and mayrequire resetting. Thus, the present disclosure is configured to processthe images quickly to avoid such discontinuities, even when theviewpoint of the portable device changes significantly in a short amountof time. Also, the present disclosure is configured to recover a lostidentification and instruct the user to return to a known viewpointwhere one or more images (e.g., stored in the memory device 14 and/ordatabase 20) can be retrieved and the process can be resumed.

After capturing the new (subsequent) image, the process 30 includesre-locating the selected cable within the new image, as indicated inblock 44. As suggested above, this may include comparing the new imagewith a previous image and analyze the differences. Assuming that thedifferences are not too significant, the processing steps can determineone or more points (or at least a portion or section) of the selectedcable from the previous view to re-locate the selected cable.

The process 30 includes a condition diamond 46 configured to determinewhether or not the selected cable has been located. If not, the process30 returns back to block 42 to repeat the capturing of a new image untilthe selected cable can be re-located. According to additionalembodiments, if the step of locating or re-locating the selected cablecannot be successfully accomplished within a certain amount of time(e.g., 10 seconds), then the process 30 may produce an error message andterminate the program. However, if the selected cable is located, theprocess 30 moves on to block 48. As indicated in block 48, the process30 includes the step of identifying one or more visible points on theselected cable. Once the identified cable is re-located on the newimage, block 48 includes the step of selecting a point within itsboundaries to serve as the start point of a new iteration of the cableidentification process.

From the identity of the selected cable and one or more points on thecable, the process 30 is able to repeat the image processing step ofblock 36 and the visualization step of block 38 continuously insubsequent time intervals for multiple images over the course of thecable following procedure. This allows the user to select the cable andthen follow the selected cable along its path. Even when the selectedcable is partially hidden from view, the image processing steps (e.g.,block 36) may be configured to continue to follow the cable and provideinstructions to the user as needed. This can assist the user and can bedone without the user trying to follow the path of the cable by himselfor herself, even if the user were to lose focus, blink, etc.

The process 30 may include a cable follower algorithm, a cableidentification algorithm, etc. and/or may be implemented in anycombination of hardware, software, and/or firmware. For example, theprocess 30 can include at least part of the augmented-realitycable-tracking program 24 shown in FIG. 1.

The process 30 may be capable of capturing images at some specific timeinterval. The high-level tasks of the process 30 may be completed in ashort delay for any given image to allow for the processing of multipleviews per second. Thus, lower priority tasks may be simplified orskipped to allow for quicker processing. This will provide the mostflexibility for visually representing images to a user with sufficientspeed and clarity. Also, the process 30 may allow for any userinteractions for subsequent steps to take place before capturing a nextimage.

Regarding block 44 of re-locating the selected cable within the newimage, the present disclosure may rely on an augmented-reality library,which may be stored in the memory device 14 and/or database 20. Thus,the process 30 can provide the functionality of displaying theidentified cable or return its coordinates on a newly taken image. Insome embodiments, the augmented-reality libraries may include known ARvision, tracking, classes, etc. offered by various vendors.

FIG. 3A is a captured image 50 related to block 32 of the process 30 ofFIG. 2. In particular, the captured image 50 may be captured as aninitial image or first image that can be used to enable user-selectionof a cable that the user wishes to follow or track. As shown in FIG. 3A,an arrow 52 or other suitable indication marker can be placed and/ormoved on the screen by user input mechanisms (e.g., tapping or slidingmotions on a display screen of a smart phone, etc.). The arrow 52 isindication of a start point related to block 34 shown in FIG. 2. FIG. 3Bis another example of a captured image 54 and a user-entered start pointarrow 56 of an initial image for allowing the user to select a cablefrom a bundle of cables, which could possibly be a tangled bunch ofcables in a relatively small space. In these examples, a first ends ofmultiple cables are terminated at connectors on a back panel of networkequipment, which may be present in a data center or other similarenvironment.

According to some embodiments, when the cable-tracking application(e.g., augmented-reality cable-tracking program 24) is first opened by auser of the portable device, a startup may run. The application maypresent a view to the user obtained from an associated image capturedevice on the portable device. The application may be configured toinstruct the user to select a start point for cable identification ofthe selected cable. This start point (e.g., one of the points related tothe arrows 52, 56) and the captured image (or a subsequent image) maythen be passed to the application to between the cable identificationprocesses.

FIGS. 4A-4E show examples of images of cables and visual highlightingfeatures related to the cable visualization step (i.e., block 38) shownin FIG. 2. In this step, the selected cable is visualized to properlyidentify this selected cable on the respective image. The visuallyhighlighted features are shown after performing the various imageprocessing steps (e.g., block 36 and/or the processes of FIG. 5). Usingthe data related to the selected cable (e.g., coordinates, thickness,color, etc.) obtained from the image processing, the application isconfigured use image visualization techniques to correctly present theselected cable on the original image.

For example, FIGS. 4A and 4B show images 58, 60, respectively, which arerelated to different examples of highlighting features added to thecaptured image 50 shown in FIG. 3A for identifying a selected cable in afirst group of cables. FIG. 4C shows image 62 related to an example ofhighlighting features added to the captured image 54 shown in FIG. 3Bfor identifying a selected cable in a second group of cables. FIG. 4D isa screen shot 64 that shows an example of the highlighting of FIG. 4Cwhile running the cable-following application on a smart phone. FIG. 4Eis a screen shot 66 that shows the captured image 54 of FIG. 3B witharchived points related to previously determined termination locationsof the second group of cables, while running the cable-followingapplication on the smart phone.

In the simplest form, the highlighted features shown in FIGS. 4A-4E mayinclude a process of stringing together a number of coordinates from oneor more cable sections determined to be part of the selected cable. Thehighlighting feature may include drawing a line on the image. Analternative highlighting feature may include using new or pre-existingaugmented-reality libraries (e.g., from one or more creators), which maybe capable of emphasizing certain identified objects or other similar ARfunctionality.

The following use cases demonstrate hypothetical situations that may beresolved by using the systems and methods described with respect to thepresent disclosure. It should be noted that these use cases are providedfor illustration purposes only to clarity the ideas and principles ofthe present disclosure and are not meant to be limiting.

Case 1: Rat-Nest Cleanup

A network technician, Information Technology (IT) engineer, or othernetwork management or maintenance personnel may be assigned toreorganize a messy cable organization (also referred to as a “ratnest”). As a first step, the technician may need to identify the path ofeach cable. Instead of attempting to visually follow each cableindividually and note their start point and end point on paper, he orshe may decide to use the augmented-reality cable-tracking program 24 orother similar application on a portable device (e.g., a smart phonebeing used by the technician).

The technician starts the application and is able to see an environmentdisplayed on a display screen of the portable device. In an initialview, the technician is able to use the application to first select acable. The application may provide an instruction, such as “select acable to follow,” “identify cable,” “cable identification mode” and/ormay select an option or menu button from a menu of possible options. Thetechnician can look at the equipment on the screen and select a cable(e.g., by touching a point on a portion of the cable on the screen,using mouse functions, joystick functions, etc.). The applicationperforms the image processing to detect as much of the selected cable aspossible and displays the selected cable (with the highlightingfeatures) on the screen. As shown in FIGS. 4A-4E, the highlighting mayinclude adding a specific color on top of the image of the selectedcable that is a different color than the other cables. The technicianmay then move the portable device (e.g., smart phone) in the directionor path of the selected cable to track this cable, while the applicationis busy identifying the selected cable from one image to the next andcontinuously highlighting the cable.

Thus, the screen shot 64 of FIG. 4D may be a sample smartphone cablefollower application. In some situations, the selected cable may not bevisible at certain points where several cables cross. In this case, thetechnician may need to move those cable out of the way to allow theapplication to track the selected cable properly. If the application hasdifficulty identifying the cable when it gets lost behind other cables,the application may instruct the user to move the other cables asnecessary to pick up the path of the selected cable again. Theapplication can then display the progression of the cable.

In some cases, the technician may wish to follow a cable that goesthrough a tube from one room to another or from one floor to another. Inthis example, the technician may be instructed by the application topause the tracking procedure and allow the technician to call acolleague in the other room or on the other floor. Using a linkedportable device, the colleague may be able to visually identify theselected cable which may temporarily be out of view from any suitableviewpoint that the portable device could use. The two technicians inthis case may then find the correct cable (e.g., by pushing/pulling thecable). Then, the second technician may start the same cable-trackingapplication (e.g., augmented-reality cable-tracking program 24) on hisor her side to join the search. In some cases, the applications runningon both devices may be configured to communicate with each other toshare useful information to allow the continuance of the cable-trackingprocess. The application on the second portable device may also allowthe second user to identify the selected cable as mentioned above withrespect to the initial selection processes (e.g., allowing the user totouch the image of the selected cable on the display screen).

In other situations, a second technician may not be available to helpthe first technician when the first technician reaches a roadblock(e.g., cables running from one room and/or floor to another). In thiscase, the technician may be able to visually note a selected cable on anopposite side of a wall or floor, pause the application, and restart theapplication after going to the other room. The technician may touch areset button on the application to start the select process again andcontinue the identification process.

When an end of the cable is identified (either at a start or an end ofthe cable-tracking process), the application may allow the user to entera long key/button press on the selected cable displayed on the screen toconfirm that the cable has been properly identified. Also, theapplication may give the user an option to label the selected cable oradd other notes about the selected cable. In one example, the engineermay add notes of “GPON ONU A23 to PPP F56” or other notes, which can bestored in the application with respect to the cable. After completion,the cable is now identified and recorded.

The application may also be accessed using the portable device or othercomputer system. Then, with the recording of information with respect tomultiple cables, the technician may be able view the cable informationfor use at a later time. In this manner, the technician can look at theequipment through the application and see what cables were identifiedand which ones were not. Once the relevant cables have been identified,later maintenance or cable replacement actions can be simplified withthe known cable tracking information. As a result, the augmented-realitycable-tracking program 24 can simplify the tasks of the technician andcan save the technician valuable time and frustration from having tofollow cables with the AR assistance. Also, the augmented-realitycable-tracking program 24 can be used to reduce the number of humanerrors that can occur from manual cable tracking processes.

Again, the engineer selects one cable by touching it on the screen.Using augmented reality, the application adds highlighting features tothe selected cable. While the engineer moves the camera or the cables inspace, the application dynamically continues to recognize the cable andupdate the highlighting on the new images. The engineer can then followthe path of the cable with the smartphone.

If the cable passes behind one or more other cables, a line-crossingimage processing algorithm (e.g., related to block 36 shown in FIG. 2)may be configured to recognize a continuity of the cable nonetheless. Ifthe cable is invisible by the camera at some point along its pathbecause it is hidden by other cables crossing its path, the engineer canmove those cables and the application will be able to update theaugmented reality to show the path now revealed.

From this data, the application may be configured to build a database ofidentified cables and their connections. Later on, in a “cable viewer”mode, the application may indicate on live images which cables werepreviously identified and which were not. The engineer can then look atthe equipment (e.g., cables and connectors) through the application tosee this information directly on screen. The application may alsodisplay the average time it took to identify the cables and a timeestimate to cover the other cables that remain to be identified. Thisfeature allows the engineer to better plan how many cables remain to beidentified, their lengths, a time estimate for tracking, and a downtimeplan to replace those cables in a cleaned and ordered configuration.

Case 2: Replace an Intermittent Defective Cable

In this next scenario, the cable tracking systems, methods, andapplications of the present disclosure may be configured to assist atechnician when a defective cable needs to be replaced. Using thepresent embodiments, a cable can be replaced with less down time thanmanual cable-tracking processes. Instead of using conventionalstrategies (e.g., recording cable connectivity information on papernotes), the network operator, technician, IT engineer, or other networkor data center worker may decide to utilize the program/applicationdescribed in the present disclosure. In some cases, the application canbe used after the first set of procedures for originally tracking andmaintaining cable connectivity information, which again can be assistedby the augmented-reality cable-tracking program 24 as describedthroughout the present disclosure for identifying and archiving one ormore of the existing cables and connections.

The technician starts the application and selects the “IdentificationArchives” mode. For example, a screen (e.g., screen shot 66 of FIG. 4E)may be shown. The technician can see the cable bundle environment (e.g.,using a camera or other image capture device of the portable device 10).In the “Identification Archives” mode, the application may provide menubuttons to allow the user to select different options. The applicationalso shows the already identified cables and/or connectors. In FIG. 4E,the connectors (e.g., terminations, ports, etc.) for connecting to oneend of the cables may be identified. The application may add distinctcolors to the identified connectors.

The technician may select a cable or connector. In this example, thetechnician is provided options to select one of multiple connectorsshown on the screen. Selection can be made by touching the screen on theconnector or a marker (e.g., circle) representing the connector. Theapplication may be configured to provide optional notes in a button-freemessage box (e.g., tips) over the selected cable, which may be usefulfor confirming that this connector is related to the target cable. Aconnector may be identified as a first end of the cable. Since the otherend of the cable may not be visible on the screen at this first stage,the application may display an arrow to help the technician orient theportable device to find the cable's destination or path.

After the phone is properly oriented, the technician may see a blurredidentification where the cable's end should be. This may be anintentional function of the application of the present disclosure toindicate that the cable end is not in a direct line of sight and may behidden by other equipment or may run in to another room. Guided by theapplication, the technician may walk into another isle (e.g., of a datacenter) and find the end of the cable identified on the smartphone'sscreen. If there are transmission issues with the cable, the technicianmay propose a cable replacement to resolve the issues, even if both endsof the cable are properly connected and do not show visible damage. Whenremoved, a new cable can be properly routed between both sets ofequipment. Connection downtime to disconnect the old cable and reconnectthe new can be reduced using this process (e.g., down to a few minutes).Thus, with the assistance of the augmented-reality cable-trackingprogram 24, the cable replacement process can be simplified for thetechnician. This can reduce human time and effort and provide moreaccurately in following a cable from one end to the other.

FIG. 5 is a flow diagram illustrating an embodiment of a sub-routine 70for identifying a cable. For example, the sub-routine 70 may be an imageprocessing analysis routine and may represent all or part of block 36shown in the process 30 of FIG. 2. The block 36 indicates the step ofidentifying a selected cable within an image (either the initial imageor any subsequent images). The cable identification sub-routine 70 ofFIG. 5 includes a first step of simplifying the cables shown in theimage to “lines” (or line segments) using an image processing technique,as indicated in block 72. The sub-routine 70 further includes the stepof identifying a first cable section from the start point (or otheridentified point) on the selected cable, as indicated in block 74.

The sub-routine 70 may be configured to proceed in one or bothdirections along the length of the selected cable. According to otherembodiments where other types of objects are tracked whereby the objectsmay include more than two directions, the sub-routine 70 may repeat thefollowing steps as needed to cover all applicable directions. However,for the sake of simplicity, the present disclosure is directed to thetracking of cables, which of course have a first end, a second end, andtwo different directions along the cable (toward the first end or towardthe second end).

As indicated in block 76, the sub-routine 70 includes proceeding firstin a first direction (from the start point) and processing a next cablesection (in the first direction) within the image. For example, thefirst direction may be oriented “upward” along the view of the cable(e.g., shown closer to the top portion of the screen of the smartphone). When repeated a second time, the block 76 can indicate theprocessing in a second direction with respect to the start point toprocess a next cable section in the second direction. The seconddirection in this case may be a “downward” direction along the cable(e.g., toward a bottom portion of the screen of the smart phone).

The sub-routine 70 further includes the step of determining if the nextcable section has been identified, as indicated in condition diamond 78.If not, the sub-routine 70 goes back to block 76 to process the nextcable section. If this cable section is identified, the sub-routine 70goes to block 80, which indicates the step of determining how the cablesection ends. For example, it may be determined that a definitive end ofthe cable is shown in the view of the captured image. The end of thecable shown within the frame or view of the capture image a) may be atermination into a connector, b) may be a sliced cable end, c) may leadout of the frame and would need be recalculated as the portable deviceis moved in the direction of the point at which the cable leads out ofthe frame, or d) may be indeterminate between of too many cables orother objects blocking the view of the cable, and/or otherpossibilities.

If a single candidate cable section cannot be decisively identifiedusing the above criteria, the block 80 of the sub-routine 70 may beconfigured to determine how the current cable section ends. The step ofblock 80, for example, may include determining if the cable isterminated, if it is plugged into a device, if it has been cut, etc. Theend can be determined if the current cable section has no candidates inthe determined ideal region. Also, the sub-routine 70 may determine ifthe image border has been reached. This occurs if the current cablesection ends at any image border. It may also be determined that theview of the cable might be too chaotic, and the identification processmight determine that multiple candidate cable sections may satisfycertain path following criteria.

Furthermore, the sub-routine 70 includes determining if both (or all)directions have been analyzed, as indicated in condition diamond 82. Forexample, if the first direction is upward, the sub-routine 70 may thenprogress in the opposite direction (e.g., downward). In otherembodiments, a cable being followed may curve to a certain extent suchthat a first direction may be upward and an opposite direction may leadto the right side (or left side) of the screen. If it is determined incondition diamond 82 that both directions have not yet been analyzed,then the sub-routine 70 returns back to block 76 to process the nextcable section for the opposite direction. After both directions havebeen analyzed, the sub-routine 70 includes the step of joining theidentified cable sections into a single cable, as indicated in block 84,and then ends.

FIGS. 6-12 show images of the example of the captured image shown inFIG. 3A during image processing techniques according to variousembodiments of the present disclosure. For example, certain imageprocessing techniques with respect to the step of simplifying cables tolines (e.g., block 72 shown in the sub-routine 70 of FIG. 5) areillustrated in FIGS. 6-9. Certain image processing techniques withrespect to the step of identifying the first cable section from thestart point (e.g., block 74 shown in the sub-routine 70 of FIG. 5) areillustrated in FIG. 10. Also, identifying next cable sections (e.g.,blocks 76 and 78) may be represented by FIG. 11. Joining cable sections(e.g., block 84) may be represented by FIG. 12.

FIG. 6 is an image 90 showing an example of a color filtering step,which may be related to the step of simplifying cables to lines (e.g.,block 72) of the sub-routine 70 of FIG. 5. The color filtering techniqueas shown in FIG. 6 may use any number of new or existingimage-processing techniques to convert a captured image of cables intolines, which can be used to simplify the cable identification process.For example, color filtering may be configured to filter out cables fromthe image which are not in the color-range of the selected cable.

FIG. 7 is an image 92 showing an example of Canny-edge detection,dilation, and erosion steps, which may be related to the step ofsimplifying cables to lines (e.g., block 72). The processes may includeconversion of the image to black and white (or a grayscale conversion).This may be done to increase the efficiency of image-processingfunctionalities. The image processing may include Canny-edge detectionand/or morphological processes (e.g., dilation process, erosion process,etc.). The process may convert overlapping cables as individualsections.

FIG. 8 is an image 94 showing an example of a skeletonization step,which may be related to the step of simplifying cables to lines (e.g.,block 72) of the sub-routine 70 of FIG. 5. FIG. 9 is an image 96 showingan example of a corner deletion step related to the sub-routine 70. Theskeletonization and corner deletion steps are configured to convertcable sections as individual lines.

FIG. 10 is an image 98 showing an example of line and point detectionsteps, which may be related to block 74 of the sub-routine 70 of FIG. 5.The image processing may include identifying a first cable section froma provided start point. As a first step, all cable sections are located.This can be done using new or existing image processing techniques(e.g., blob detection, etc.). Then, using the image coordinates of theuser's selection, the image processing technique may include identifyingthe closest cable section via a simple comparison of points. In FIG. 10,located cable sections are represented by yellow lines & blue dots. Thecable section selected by the user is shown in red.

FIG. 11 is an image 100 showing an example of a cable section trackingstep, which may be related to blocks 76, 78 of the sub-routine 70 ofFIG. 5. The cable section tracking step may include identifyingsubsequent cable sections in one direction. For example, theimage-processing technique as shown in the image 100 of FIG. 11 may berepeated for both cable directions. In may be noted that cableidentification may be performed in both direction (e.g., upward anddownward directions) to search for the next cable section. Thedescription below is directed to only one direction (e.g., the downwarddirection) for the sake of brevity, but the same processes may beperformed in the opposite direction as well to yield an identified cableto be visualized.

Identify the next cable section may include a first step of selecting,from the list of all located cable sections, a subset (e.g., the top 5)of sections that are closest in distance to the bottom edge of theoriginal cable section. A second step may include defining an idealregion for the next cable section (e.g., a triangular shaped regionextending downwards from the current cable section). To determine inwhich direction the triangle should extend, the following sub-steps maybe taken:

-   -   Take two points of the current cable section (e.g., one near the        bottom, and one far enough towards the top to accurately        determine a “slope”);    -   Compute a slope from these two points; and    -   Assuming a triangular region is desired (other region shapes        could also be used), define the region starting from the bottom        edge of the current cable section and extend in the determined        direction using positive and negative angles off the computed        slope (e.g., 15 degrees off the slope).

The next step for identifying the next cable section may includeselecting, from the list of the closest candidates, the one candidatethat best satisfies the following criteria:

-   -   The candidate falls within the ideal region defined above.    -   The candidate's computed slope (using the same technique as        above but starting from its top edge) must intersect with the        current cable section's computed slope.    -   The intersection angle between the two cable sections must be as        close to 180 degrees as possible.

The image 100 of FIG. 10 depicts the current cable section in red, thecenter points of each candidate cable section as yellow dots, and theideal region (triangular in shape) with blue lines. The image processingprocess then include repeating the above process steps for eachsubsequent cable section until no more sections can be identified.

FIG. 12 is an image 102 showing an example of a cable section joiningstep, which may be related to block 84 of the sub-routine 70 of FIG. 5.Once all cable sections have been identified in both directions, theprocess of joining the identified cable sections into a single cable mayinclude combining the data (e.g., coordinates, thickness, color, etc.)of the cable sections into a single cable entity. This grouping allowsfor future cable-specific data to be associated to the cable entitylater on (e.g., start connector, end connector, service provider info,etc.).

In addition to the various embodiments of the systems and methods fortracking a cable amidst a bunch of cables and identifying a selectedcable from one view to a next, the present disclosure further includesalternative cable identification solutions, which may provide additionalimprovements. For example, a first change may include a process torestrict cable identification process to a focused region of each of thecaptured images. Therefore, the application can focus a region, whichcould either be provided by the user or estimated automatically from asimpler rough cable detection algorithm.

Another modification may include the use of a depth camera (or the useof many images from different vantage points) to build a 3D map ofbundle of cables. The addition of depth analysis could improve theability of the present systems and methods to handle cable bundles,knots, and other such complexities, which may be more difficult to modelin 2D.

Also, another alternative solution may include detecting a cable usingmultiple tries, attempts, or iterations. This could be done usingpreviously acquired images or by having the user manually moving cablesand use the detected cable information from each image to improve theability of the present systems and methods to handle cable bundles,knots, and other such complexities.

In addition, further improvements for the cable-tracking systems andmethods of the present disclosure may include: a) adding cable maskingtechniques to the image processing stages, b) replacing the processingdevice 12 with a dedicated Graphical Processing Unit (GPU), which may becapable to faster and more efficient image processing techniques and mayimprove image and signal processing, and c) performing a higher degreeof approximation to replace linear approximation processes to detect thenext best candidate with higher approximation processes to yield abetter estimate concerning curved cables.

An additional improvement may include forging all routes from an initialglobal picture. For example, before starting any cable identification,the application may invite the user to take one or several pictures toprovide a global picture. This may help to pre-define routes and helpdiscriminate the selected cable from others in a part of the picturewhere distinction is harder.

Also, another improvement may include preserving key information takenfrom the initial image (e.g., cable color, boundaries, etc.) to improvethe later tasks of the cable identification process. For example, theimage processing steps tend to remove details from the image as itadvances. This improvement would seek to make those key detailsavailable to later tasks of the cable identification process.

Alternative cable identification methodologies may include the followingalternative solutions to identify cables within an image. For example,cable identification may include the use of Machine Learning (ML). Inthis case, the application may define and train a ML model (or utilize amode processed in advance) to handle the identification of cablesections. This may include continuously collecting identification datafrom users to update and re-train the model. This would allow all usersto benefit from a ML model that evolves and improves over time.

Furthermore, the embodiments of the systems and methods of the presentdisclosure may also include complementary functions to improveusability, as described as follows. The additional functionality mayinclude the ability to persist and quickly re-identify cables identifiedduring previous sessions. The application may be configured with theability to attach meta data to identified cables (e.g., source andtarget connector information, service provider information, etc.) andinclude the meta data in the identified cables persisted data. Othercomplementary functions may include organizing already identified cablesby room and equipment. Also, the application may export persisted datafor identified cables in multiple formats (e.g., raw data, pictures,spreadsheets, etc.).

The present embodiments may include other complementary functions, suchas cloud-sharing of already identified cables between many users inreal-time. Also, the application can superimpose visual cues on theimage to help guide the user to the next step (e.g., help the userre-orient the device camera when a previously identified cable cannot bere-identified on the current image). The application may allow the userto pause the cable identification process to better understand what hasbeen identified up to that point. Also, the application can alert theuser if a previously identified cable can no longer be found on animage.

FIG. 13 is a flow diagram illustrating an embodiment of another process110 for visually tracing a selected cable using augmented reality. Asillustrated in this embodiment, the process 110 includes capturing anddisplaying an image of a bundle of cables, as indicated in block 112.For example, the image captured by the camera may be rendered on a smartdevice. At this stage, no augmented reality feature is included to therendered image if a cable has not yet been selected, if the cable or itsintermediary position is out-of-sight, or if an error is detected duringthe analysis.

The process 110 also include allowing the user to select a cable, asindicated in block 114. The step may include waiting for a response fromthe user, wherein, upon the user selecting one cable (e.g., by clickingon the screen), the application moves to the next step. Also, theprocess 110 includes using an augmented reality library to identify aselected cable in the image, as indicated in block 116. The image may beprocessed in augmented reality to select the cable path on the pointclicked by the user (i.e., the selected cable path corresponding to theselected cable). The selected cable start-up position may be stored as a3D coordinate to be converted to 2D coordinates for the image analysis.These coordinates may also be provided by an augmented reality library.The 3D coordinates on the discovered cable path may be temporarilystored in order to allow to continue image analysis whenever the cable'sstart position becomes out-of-sight.

Furthermore, the process 110 also includes converting the image to linesfor line analysis, as indicated in block 118. Also, the process 110includes filtering out the non-selected lines as much as possible, asindicated in block 120. The image may be converted to lines definingcable paths and non-selected lines are filtered.

Then, the process 110 includes performing a line-crossing analysis toaccount for cables that cross paths within the particular viewpoint ofthe captured image, as indicated in block 122. The process 110 alsoincludes the step of keeping a line that has the least amount of anglechange (e.g., the straightest line), as indicated in block 124. In thisstep, line crossing analysis may be performed on a 2D line image withoutany deepness information. Line segments may be added to the selectedcable path if it forms an angle within a predetermined scope (e.g., plusor minus 10 degrees) with the selected cable path.

In addition, the process 110 includes the step of continuing lineanalysis until a line termination is found or the path cannot be furtherdetected, as indicated in block 126. If, at the end of the selectedcable path, any lines are found within that predetermined angles (e.g.,10 degree margin), these lines can then be considered as possible lineendpoint. Block 128 indicates a step of superposing virtual features tothe image based on the line analysis. For example, this step may includeadding color highlighting to the rendered image along the selected cablepath to visually emphasize or identify the selected cable.

Also, the process 110 may include determining if an error is detected,as indicated with respect to condition diamond 130. Error detection mayinclude determining if an error is detected after any or all of thesteps described with respect to blocks 116, 118, 120, 122, 124, 126,128. If an error was detected with respect to condition diamond 130, theprocess 110 ends. If no error is detected, the process 110 includescapturing and displaying a new image, as indicated in block 132, andlooping back to block 116 to repeat the cable identification proceduresas desired. Thus, the process 110 can be repeated in real time for eachnew image captured to provide a real time effect. If the user moves thecables, the process 110 continues to update the definition of theselected cable path in real time.

Some features of the systems and methods of the present disclosure mayinclude using augmented reality to help a worker to dynamically followthe path of a selected cable among a rat nest of multiple cables byhighlighting on a screen the path of the selected cable. The presentembodiments may also be configured to distinguish the selected cableamong a rat nest of cables using line-crossing analysis on the 2D image.The present disclosure also includes the feature of dynamically trackinga cable when camera orientation changes and cables are moved and keepingtrack of the cable path even if its original point is lost.

FIG. 14 is a flow diagram illustrating another embodiment of a process140 for visually tracing a selected cable using augmented reality. Theblocks shown with solid lines may be considered to be main features ofthe present disclosure, while the blocks shown with dashed lines may beconsidered to be extra features. The process 140 includes a first stepof obtaining an initial captured image showing a bundle of cables, asindicated in block 142. In some embodiments, the process 140 mayoptionally include receiving a user input for identifying a selectedcable of the plurality of cables shown in the initial captured image, asindicated in block 144. The process 140 also includes processing theinitial captured image to distinguish the selected cable from othercables of the bundle of cables, as indicated in block 146. Also, theprocess 140 includes displaying the initial captured image on a displayscreen while visually augmenting an image of the selected cable tohighlight the selected cable with respect to the other cables, asindicated in block 148.

According to some additional optional steps, the process 140 may furtherinclude determining whether or not a command is received (from the user)to exit the program, as indicated in condition diamond 150. If the userwishes to stop, the process 140 comes to an end. If no input is receivedto stop the process 140, a step of obtaining a next captured image isperformed, as indicated in block 152. Then, the process 140 includesusing a tracking technique to identify (re-locate) the selected cable tomaintain continuity, as indicated in block 154. At this point, theprocess 140 loops back to block 146 to repeat the image processingfunctions and continuing the tracking of the selected cable.

In some embodiments, additional optional features may further include aprocess of freezing (pausing) image analysis. When the cable-trackingprocedure can be resumed at a later time when the user places theportable device in the same or similar viewpoint as the place when theprocess was paused.

The systems and methods may also include the possibility of storingmultiple cable configurations (e.g., for several server rooms, isles, orother areas within a predetermined space such as a data center, etc.).Also, the present embodiments may include cloud-sharing functionality(e.g., using the external interface 18) to share cable configurationsamong several user devices. In some cases, the shared information may beshared in real-time or a delayed sharing.

Also, the present disclosure may be configured to allow exportation ofcable configurations in multiple formats (e.g., native data, pictures,spreadsheets, etc.). The present systems and methods may alsodistinguish on the user screen the identified cables from thenon-identified ones. The present disclosure may also provide a timeestimate to identify a cable and others.

One purpose or goal of the present disclosure may be to offer a tool tominimize downtime due to cable maintenance tasks. This can beimprovement over manually following cables according to conventionalprocesses. Some other objectives and technical advantages may includebeing able to properly detect a cable, even if it is interlaced ortangled up among other cables. Also, the present disclosure may be ableto follow or relocate the detected cable among moving images or movingcables. Therefore, the present systems and methods may provide betterthan human precision and efficacy.

The solutions provided by the present disclosure may include systems,methods, non-transitory computer-readable media, applications, computerprograms, portable devices (e.g., smart phones, tablet computers),computer systems, data capture and display systems, and other varioussystems. The various embodiments may include an application usingaugmented reality to identify and follow a designated cable. When acable is selected, the program or application may be configured toperform a process of virtually distinguishing the selected cable fromthe others by adding colors, animations, or other computer-generated orAR features on the smart device rendered image.

If the cable destination is not visible on screen, the application maybe configured to orient or direct the user towards the properdestination. Directing the user may include providing an aerial view,showing see-through images, displaying arrows on the screen, or othersuitable instructional information.

In some embodiments, the AR techniques and algorithms described in thepresent disclosure may be developed as a software prototype or algorithm(e.g., the augmented-reality cable-tracking program 24). The program orapplication described in the present disclosure focuses mainly on thefirst part of identifying the cable in one image (i.e., the first orinitial image). However, it should be noted that the additionalsubsequent images may be rendered in a similar manner using the varioussteps for each image. Thus, the display screen may show a currentviewpoint or position as the portable device is moved in a continuousmanner. The screen is refreshed with subsequent images accordingly.

As described above, the process 30 may be capable of capturing images atsome specific time interval. Also, there is a need to implement theprocess 30 in real-time or in pseudo-real-time. As described herein,pseudo-real-time means a user perceives real-time execution. Based onexperimentation, it was determined that the process 30 should processabout 30 images per second or about one image every 33 milliseconds.

Additionally, users typically tolerate an upfront latency beforedisplaying the first image. This latency could be used to perform thealgorithm steps that would not need to be re-performed for each capturedimage. Using a parallel processing strategy, like a pipeline, coulddrastically minimize the latency between displayed images. An acceptableupfront latency is typically defined as less than 300 milliseconds (ms).

Based on experimentation, a prototype was developed that provided thefollowing numbers:

Step Delay, in mS Step Description 1 1 Load image, Color filtering 2 6Black and White Conversion 3 55 Canny edge, Dilation, Erosion 4 13 Imagepreparation, Skeletonization 5 67 Corner detection 6 44 Cable detectionalgorithm 7 3 Complete process and report results Total 189 ms

To display an image every 33 ms with a 190 ms process, the algorithmwould need to be broken into a parallel pipeline of 6 steps.

Several optimizations were proposed herein. Specifically consider the“Restrict cable identification process to a focused region of the image”suggestion and assume that the image analysis and processing could belimited to a ¼ of the original image. One could then roughly estimatethe total processing time would decrease to 142/4+44/2+3=61 ms. At thatspeed, a simpler parallel pipeline of only 2 or 3 steps would be enoughto provide a perceived real-time user experience.

The above measurements and example optimization demonstrate that aperceived real-time user experience is achievable with this algorithm.Even without optimization, the algorithm could use the describedparallel processing pipeline strategy to display images at 33 ms with anupfront latency of 189 ms, lower than the suggested 300 ms upper limit.If we consider the optimization to limit the processing to a region ofinterest, we could go as low as 61 ms latency, giving us a margin forlower pipeline, better latency, lower processing power or less memoryrequirements.

Although the present disclosure has been illustrated and describedherein with reference to various embodiments and examples, it will bereadily apparent to those of ordinary skill in the art that otherembodiments and examples may perform similar functions, achieve likeresults, and/or provide other advantages. Modifications, additions, oromissions may be made to the systems, apparatuses, and methods describedherein without departing from the spirit and scope of the presentdisclosure. All equivalent or alternative embodiments that fall withinthe spirit and scope of the present disclosure are contemplated therebyand are intended to be covered by the following claims.

What is claimed is:
 1. An Augmented Reality (AR) method comprising thesteps of: obtaining an initial captured image showing a bundle ofcables; processing the initial captured image to distinguish a selectedcable from other cables of the bundle of cables using a trackingtechnique, including a line-crossing analysis, to identify sections ofthe selected cable among the bundle of cables and keep track of theselected cable even when points along the selected cable are lost withinthe initial captured image, the line-crossing analysis including addingline segments to a path of the selected cable in response to the linesegments forming an angle with the path that is within a predeterminedangle; displaying the initial captured image on a display screen whilevisually augmenting an image of the selected cable to highlight theselected cable with respect to the other cables; determining an end ofthe selected cable is missing from the initial captured image; anddisplaying an orientation marker responsive to determining that the endof the selected cable is missing from the initial captured image.
 2. TheAR method of claim 1, further comprising the steps of: receiving a nextcaptured image after processing the initial captured image; comparingthe next captured image with the initial captured image; using thetracking technique to identify at least a section of the selected cableamong the bundle of cables in the next captured image; distinguishingthe selected cable shown in the next captured image from the othercables; updating the display screen to display the next captured imagewhile visually augmenting a next image of the selected cable tohighlight the selected cable with respect to the other cables; repeatingthe receiving, comparing, using, distinguishing, and updating steps oneor more times relative to a most recently processed captured image untilone of a line termination is found and the path cannot be furtherdetected, and in response to the path that cannot be further detected,identifying any lines within the predetermined angle to be considered asa possible endpoint.
 3. The AR method of claim 2, wherein comparing thenext captured image with the initial captured image includes determiningdifferences between the next captured image and the initial capturedimage when an image capture device captures the next captured image andthe initial captured image from different viewpoints or when the atleast one cable of the bundle of cables is moved.
 4. The AR method ofclaim 1, wherein the processing technique further includes at least oneprocedure selected from an object recognition procedure, a pointdetection procedure, a color filtering procedure, a Canny-edge detectionprocedure, a morphological procedure, a dilation operation, an erosionoperation, a skeletonization procedure, and a corner deletion procedure.5. The AR method of claim 1, wherein processing the initial capturedimage includes tracking each cable of the bundle of cables.
 6. The ARmethod of claim 1, wherein processing the initial captured imageincludes identifying depth information for each cable of the bundle ofcables and mapping each cable of the bundle of cables in athree-dimensional map responsive to the identified depth information. 7.The AR method of claim 1, wherein obtaining the initial captured imageincludes receiving the initial captured image from a camera of aportable user device, the display screen being associated with theportable user device.
 8. The AR method of claim 7, wherein the portableuser device is one of a smart phone, a tablet, eyewear, and a head-updisplay.
 9. The AR method of claim 1, further comprising the step ofreceiving a user input for identifying the selected cable of the bundleof cables.
 10. The AR method of claim 1, wherein visually augmenting theimage of the selected cable includes the step of superposing virtualimage features on the image of the selected cable.
 11. The AR method ofclaim 10, wherein superposing virtual image features includes the stepof adding color or animation features to the image of the selectedcable.
 12. The AR method of claim 11, wherein visually augmenting theimage of the selected cable further includes displaying one or morevisual features for showing portions of the selected cable that arehidden from view.
 13. The AR method of claim 11, further comprising thestep of using machine learning to distinguish the selected cable fromthe other cables.
 14. A portable computing system comprising: an imagecapture device, a display screen, a processing device, and a memorydevice configured to store a computer program having instructions that,when executed, enable the processing device to obtain an initialcaptured image showing a bundle of cables from the image capture device,process the initial captured image to distinguish a selected cable fromother cables of the bundle of cables using a tracking technique,including a line-crossing analysis, to identify sections of the selectedcable among the bundle of cables and keep track of the selected cableeven when points along the selected cable are lost within the initialcaptured image, the line-crossing analysis including adding linesegments to a path of the selected cable in response to the linesegments forming an angle with the path that is within a predeterminedangle, cause the display screen to display the initial captured imagewhile visually augmenting an image of the selected cable to highlightthe selected cable with respect to the other cables; determine an end ofthe selected cable is missing from the initial captured image; and causethe display screen to display an orientation marker responsive todetermining that the end of the selected cable is missing from theinitial captured image.
 15. The portable computing system of claim 14,wherein the portable computing system is one of a smart phone, a tablet,eyewear, and a head-up display.
 16. The portable computing system ofclaim 14, wherein visually augmenting the image of the selected cableincludes superposing virtual image features on the image of the selectedcable, wherein superposing virtual image features includes adding coloror animation features to the image of the selected cable, and whereinvisually augmenting the image of the selected cable further includesdisplaying one or more visual features for showing portions of theselected cable that are hidden from view.
 17. A non-transitorycomputer-readable medium configured to store computer logic havinginstructions that, when executed, cause one or more processing devicesto: obtain an initial captured image showing a bundle of cables; processthe initial captured image to distinguish a selected cable from othercables of the bundle of cables using, a tracking technique including aline-crossing analysis, to identify sections of the selected cable amongthe bundle of cables and keep track of the selected cable even whenpoints along the selected cable are lost within the initial capturedimage, the line-crossing analysis including adding line segments to apath of the selected cable in response to the line segments forming anangle with the path that is within a predetermined angle; display theinitial captured image on a display screen while visually augmenting animage of the selected cable to highlight the selected cable with respectto the other cables determine an end of the selected cable is missingfrom the initial captured image; and display an orientation marker onthe display screen responsive to determining that the end of theselected cable is missing from the initial captured image.
 18. Thenon-transitory computer-readable medium of claim 17, wherein theinstructions further cause the one or more processing device to: receivea next captured image after processing the initial captured image;compare the next captured image with the initial captured image; use thetracking technique to identify at least a section of the selected cableamong the bundle of cables in the next captured image; distinguish theselected cable shown in the next captured image from the other cables;update the display screen to display the next captured image whilevisually augmenting a next image of the selected cable to highlight theselected cable with respect to the other cables; repeat the receiving,comparing, using, distinguishing, and updating one or more timesrelative to a most recently processed captured image until one of a linetermination is found and the path cannot be further detected, and inresponse to the path that cannot be further detected, identifying anylines within the predetermined angle to be considered as a possibleendpoint.
 19. The non-transitory computer-readable medium of claim 18,wherein comparing the next captured image with the initial capturedimage includes determining differences between the next captured imageand the initial captured image when an image capture device captures thenext captured image and initial captured image from different viewpointsor when the at least one cable of the bundle of cables is moved.
 20. Thenon-transitory computer-readable medium of claim 17, wherein theprocessing technique further includes at least one procedure selectedfrom an object recognition procedure, a point detection procedure, acolor filtering procedure, a Canny-edge detection procedure, amorphological procedure, a dilation operation, an erosion operation, askeletonization procedure, and a corner deletion procedure.